Share vitest workers for node-environment tests and cut fixed per-file costs - #2303
Merged
SawyerHood merged 3 commits intoAug 22, 2026
Merged
Conversation
…e costs Re-importing the module graph for every test file was 80-90% of the big suites' CPU. vitest.shared.ts now exports sharedWorkerProjects(), which splits a package's tests into shared-worker projects (isolate: false) and an isolated project for files that mutate worker-global state (vi.mock, stubs, process.env, global assignments, including through test helpers) or run in a DOM environment. A SharedWorkerSequencer orders the queue so vitest actually reuses workers. Tests alias @hugeicons/core-free-icons to its single-file bundle (45ms instead of ~700ms per worker), and the server harness clones a migrated SQLite template instead of replaying 107 migrations per test. Co-Authored-By: Claude <noreply@anthropic.com>
…nd Pi suites The templates external-scaffold test ran four cold `npm install`s of the packed SDK (~13s each); it now installs once per file, symlinks every scaffold's node_modules at that install, skips audit/fund round trips, and overlaps the backend and frontend typechecks. db data suites open a migrated in-memory template instead of replaying 107 migrations per test. The integration harness memoizes the first-party bridge artifact build per worker (~0.6s of esbuild per test). provider-parity raises maxConcurrency to 16: each replay cell is a bridge child that mostly waits on pacing, so the suite was wall-clock bound by vitest's default of 5 (37s -> 15s on 4 CPUs). Two Pi SDK retry tests drive the 250ms retry delay with fake timers instead of sleeping through eight of them. Co-Authored-By: Claude <noreply@anthropic.com>
Packages whose tsconfig covers their vitest config typecheck vitest.shared.ts with exactOptionalPropertyTypes, which rejects passing an undefined value to an optional property. Co-Authored-By: Claude <noreply@anthropic.com>
SawyerHood
deleted the
bb/optimize-test-suite-performance-thr_2ezguwcmef
branch
August 22, 2026 16:09
lnittman
added a commit
to lnittman/bb
that referenced
this pull request
Aug 22, 2026
…id a spine Four surfaces, one argument: the page states that bb builds itself and then never shows you which work was bb's. **The feed now marks it.** `agentMergedLastMonth` was already being measured — a search for the "AGENT GENERATED" line the repo requires of agent-created PRs — and then never rendered. The same query now also writes pr-feed.json, so each row carries an `agent` flag from the identical source the aggregate is counted from and the two can never disagree. Both searches had to be ordered by creation date first: at the default best-match sort their pages shared almost nothing, which scored a feed of zero agent PRs against a 60% aggregate. Seventeen of the last eighteen are agent-written, verified against the real bodies of get-bb#2303, get-bb#2274 and get-bb#2267, with get-bb#2258 correctly unmarked. The marker is deliberately quiet. On seventeen of eighteen rows it is not a differentiator, it is a column the eye reads straight down — and the one row without it is what makes the rest credible. **The stat block trades forks for it.** 436 sits beside 721 so the adjacency does the arithmetic without a percentage. The fork count said nothing a reader could act on; this is the headline restated as a measurement. **The bento gets spans.** Ten columns instead of two halves, 6-4 / 4-6. Subagents leads wide because the tree needs room for its indent guides to mean anything, while a diff hunk is narrow by nature and reads fine at four. Uniform halves were also why the phones sat in cards built for a window. The heading becomes a chip, which stops it competing with the app's own headings inside the mock, and `text-wrap: nowrap` goes — it was truncating every description that did not happen to fit, a promise unkeepable once the tiles stopped being the same width. Three stacked shadows replace the single blur. **The footer becomes an index.** Seven links on one middot-separated line was a run-on sentence with 17px hit areas; they are now three named groups of 28px rows in monospace uppercase, a voice used nowhere else on the page because the footer is the one part that is pure address. Also: the closer mark rests on the riso print. It rested on bb-icon.png — the 192px macOS app icon, whose rounded tile is painted into the artwork, so the mark sat on a plaque no CSS could remove — and hid the good drawing behind a click most readers would never make. Feed tracks widen to two columns: at three, every row had to fit an avatar, a title, the marker and "get-bb#2303 · Aug 22" in 320px, and the title was what got the ellipsis.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was wrong
The test suites spent most of their time outside the tests. Profiling every package uncontended showed that 80–90% of the big suites' CPU was per-file overhead: vitest re-imported the whole module graph, built jsdom, and ran the setup file for each of
@bb/app's 422 files; the setup file's@bb/shared-ui/icon-extendedimport pulled@hugeicons/core-free-icons, a barrel that re-exports 5,122 one-icon modules and takes ~0.7 s to load; and every@bb/servertest harness replayed all 107 SQLite migrations (57 of the 61 ms a harness costs, paid by 1,922 tests).@bb/serverand@bb/agent-runtimealready setisolate: false, but vitest 4 only hands a finished worker the next queued file when that file has the same project and environment, so the interleaved queue still churned workers.Per-test profiling (JSON reporter) then showed fixed costs inside the slowest suites: the templates external-scaffold test ran four cold
npm installs of the packed SDK (~13 s each); db data suites migrated per test (~17 s of their 30 s); every integration harness rebuilt the first-party provider bridges with esbuild (~0.6 s);provider-parity's 43 replay cells are bridge children that mostly wait on settle/drain pacing, so the suite was wall-clock bound by vitest's defaultmaxConcurrencyof 5; and two Pi SDK tests slept through eight real 250 ms retry delays each.What changed
vitest.shared.tsexportssharedWorkerProjects(). It scans a package's test files and splits them into shared-worker projects (isolate: false, one per non-DOM environment) and an isolated project for files that mutate worker-global state —vi.mock/vi.stubGlobal/vi.stubEnv/vi.resetModules,process.envwrites,process.chdir, assignments towindow/globalThis/document/navigator/prototypes (including through a cast),Object.defineProperty/Reflect.*on them — directly or through a test helper they import. Files that run injsdom/happy-domstay isolated: pushing the app's jsdom files through one worker failed a different file on every ordering (portals, focus, media-query caches keyed by the shared document), which no source scan can enumerate. Every project keeps the package's ownincludeglobs and excludes the other projects' files, so the split never changes which files run.SharedWorkerSequencer(installed bydefineWorkspaceTestConfigand the root config) orders the run queue: isolated files first, then each shared project contiguously. This is what turnsisolate: falseinto worker reuse withoutgroupOrderphase barriers, which left workers idle at each phase tail.@hugeicons/core-free-iconsspecifier to the package's self-contained minified bundle (identical exports; 45 ms instead of ~700 ms per worker).vitest.config.tsfiles use the helper;@bb/serverand@bb/agent-runtimedrop their hand-rolled split.apps/server/test/helpers/test-app.tsaddscreateTestDb(), which migrates once per worker and opens each harness from the serialized image;createConnectionaccepts aBufferfor that.packages/db/test/helpers/migrated-connection.tsdoes the same for the db data suites (suites that exercisemigrateitself still call it directly).packages/templates/test/plugin-scaffold-external.test.tsinstalls the packed SDK once per file and symlinks each scaffold'snode_modulesat it, passes--no-audit --no-fund --prefer-offline, and overlaps the backend and frontend typechecks.apps/server/test/helpers/provider-registry.tsmemoizes the first-party bridge artifact build per worker process.packages/provider-parity/vitest.config.tssetsmaxConcurrency: 16(measured pinned to 4 CPUs: 37 s at 5, 21 s at 10, 15 s at 16, all green).internal-skill-trees.test.tspins the fixture's file mode (it failed underumask 002),@bb/qagets a 15 s test timeout, andAGENTS.mddocuments the convention.How you verified
Clean, uncontended runs on a 16-core machine:
@bb/app63.0 s → 38.6 s,@bb/server47.6 s → 32.0 s,@bb/mobile4.4 s → 2.4 s,@bb/provider-parity36.5 s → 14.0 s,@bb/integration-tests24.8 s → 16.6 s,@bb/db7.5 s → 3.5 s, the templates external test 27.3 s → 23.5 s with a warm npm cache (three cold installs fewer in CI),sdk-session.test.ts4.0 s → 0.3 s; other packages at parity or slightly faster.pnpm exec turbo run test --force --continue3 m 27 s → 2 m 22 s with everything contending, 71 of 72 tasks passing. The one failure,@bb/qa's "starts a detached daemon repeatedly" test, times out on the development machine before and after this change. Stress: every shared project run through a single worker (--project <name> --maxWorkers=1) passes for app (149 files), server (184), tasks, host-daemon, and cli;vitest run --shard=1/3works for the app.pnpm exec turbo run typecheckpasses for db, server, app, templates, integration-tests, provider-parity, and agent-runtime;lintpasses;tsc --noEmitpasses onvitest.shared.ts.No linked issue.